The Functional Role of Striatal Cholinergic Interneurons in Reinforcement Learning From Computational Perspective

In this study, we explore the functional role of striatal cholinergic interneurons, hereinafter referred to as tonically active neurons (TANs), via computational modeling; specifically, we investigate the mechanistic relationship between TAN activity and dopamine variations and how changes in this relationship affect reinforcement learning in the striatum. TANs pause their tonic firing activity after excitatory stimuli from thalamic and cortical neurons in response to a sensory event or reward information. During the pause striatal dopamine concentration excursions are observed. However, functional interactions between the TAN pause and striatal dopamine release are poorly understood. Here we propose a TAN activity-dopamine relationship model and demonstrate that the TAN pause is likely a time window to gate phasic dopamine release and dopamine variations reciprocally modulate the TAN pause duration. Furthermore, this model is integrated into our previously published model of reward-based motor adaptation to demonstrate how phasic dopamine release is gated by the TAN pause to deliver reward information for reinforcement learning in a timely manner. We also show how TAN-dopamine interactions are affected by striatal dopamine deficiency to produce poor performance of motor adaptation

25th annual computational neuroscience meeting: CNS-2016

The same neuron may play different functional roles in the neural circuits to which it belongs. For example, neurons in the Tritonia pedal ganglia may participate in variable phases of the swim motor rhythms [1]. While such neuronal functional variability is likely to play a major role the delivery of the functionality of neural systems, it is difficult to study it in most nervous systems. We work on the pyloric rhythm network of the crustacean stomatogastric ganglion (STG) [2]. Typically network models of the STG treat neurons of the same functional type as a single model neuron (e.g. PD neurons), assuming the same conductance parameters for these neurons and implying their synchronous firing [3, 4]. However, simultaneous recording of PD neurons shows differences between the timings of spikes of these neurons. This may indicate functional variability of these neurons. Here we modelled separately the two PD neurons of the STG in a multi-neuron model of the pyloric network. Our neuron models comply with known correlations between conductance parameters of ionic currents. Our results reproduce the experimental finding of increasing spike time distance between spikes originating from the two model PD neurons during their synchronised burst phase. The PD neuron with the larger calcium conductance generates its spikes before the other PD neuron. Larger potassium conductance values in the follower neuron imply longer delays between spikes, see Fig. 17.Neuromodulators change the conductance parameters of neurons and maintain the ratios of these parameters [5]. Our results show that such changes may shift the individual contribution of two PD neurons to the PD-phase of the pyloric rhythm altering their functionality within this rhythm. Our work paves the way towards an accessible experimental and computational framework for the analysis of the mechanisms and impact of functional variability of neurons within the neural circuits to which they belong

Braided Multi-Electrode Probes for Neural Interfaces

Although clinical use of invasive neural interfaces is very limited, due to safety and reliability concerns, the potential benefits of their use in brain machine interfaces (BMIs) seem promising and so they have been widely used in the research field. Microelectrodes as invasive neural interfaces are the core tool to record neural activities and their failure is a critical issue for BMI systems. Possible sources of this failure are neural tissue motions and their interactions with stiff electrode arrays or probes fixed to the skull. To overcome these tissue motion problems, we have developed novel braided multi-electrode probes (BMEPs). By interweaving ultra-fine wires into a tubular braid structure, we obtained a highly flexible multi-electrode probe. In this thesis we described BMEP designs and how to fabricate BMEPs, and explore experiments to show the advantages of BMEPs through a mechanical compliance comparison and a chronic immunohistological comparison with single 50µm nichrome wires used as a reference electrode type. Results from the mechanical compliance test showed that the bodies of BMEPs have 4 to 21 times higher compliance than the single 50µm wire and the tethers of BMEPs were 6 to 96 times higher compliance, depending on combinations of the wire size (9.6µm or 12.7µm), the wire numbers (12 or 24), and the length of tether (3, 5 or 10 mm). Results from the immunohistological comparison showed that both BMEPs and 50µm wires anchored to the skull caused stronger tissue reactions than unanchored BMEPs and 50µm wires, and 50µm wires caused stronger tissue reactions than BMEPs. In in-vivo tests with BMEPs, we succeeded in chronic recordings from the spinal cord of freely jumping frogs and in acute recordings from the spinal cord of decerebrate rats during air stepping which was evoked by mesencephalic locomotor region (MLR) stimulation. This technology may provide a stable and reliable neural interface to spinal cord researches with freely moving animals as well as to BMI researches. In addition this is extensible to various applications.Ph.D., Biomedical Engineering -- Drexel University, 201

The interplay between cerebellum and basal ganglia in motor adaptation: A modeling study

Motor adaptation to perturbations is provided by learning mechanisms operating in the cerebellum and basal ganglia. The cerebellum normally performs motor adaptation through supervised learning using information about movement error provided by visual feedback. However, if visual feedback is critically distorted, the system may disengage cerebellar error-based learning and switch to reinforcement learning mechanisms mediated by basal ganglia. Yet, the exact conditions and mechanisms of cerebellum and basal ganglia involvement in motor adaptation remain unknown. We use mathematical modeling to simulate control of planar reaching movements that relies on both error-based and non-error-based learning mechanisms. We show that for learning to be efficient only one of these mechanisms should be active at a time. We suggest that switching between the mechanisms is provided by a special circuit that effectively suppresses the learning process in one structure and enables it in the other. To do so, this circuit modulates learning rate in the cerebellum and dopamine release in basal ganglia depending on error-based learning efficiency. We use the model to explain and interpret experimental data on error- and non-error-based motor adaptation under different conditions

Muscle forces do not have significant directional dependence.

<p>Average forces for the six arm muscles (black) fitted with cosine tuning curves (green). The R² is relatively small, implying that muscle forces are not directionally tuned. Error bars show standard deviations across 50 trails with randomly chosen motor strategy.</p

Muscle model parameters.

<p>Muscle model parameters.</p

From the motor cortex to the movement and back again

<div><p>The motor cortex controls motor behaviors by generating movement-specific signals and transmitting them through spinal cord circuits and motoneurons to the muscles. Precise and well-coordinated muscle activation patterns are necessary for accurate movement execution. Therefore, the activity of cortical neurons should correlate with movement parameters. To investigate the specifics of such correlations among activities of the motor cortex, spinal cord network and muscles, we developed a model for neural control of goal-directed reaching movements that simulates the entire pathway from the motor cortex through spinal cord circuits to the muscles controlling arm movements. In this model, the arm consists of two joints (shoulder and elbow), whose movements are actuated by six muscles (4 single-joint and 2 double-joint flexors and extensors). The muscles provide afferent feedback to the spinal cord circuits. Cortical neurons are defined as cortical "controllers" that solve an inverse problem based on a proposed straight-line trajectory to a target position and a predefined bell-shaped velocity profile. Thus, the controller generates a motor program that produces a task-specific activation of low-level spinal circuits that in turn induce the muscle activation realizing the intended reaching movement. Using the model, we describe the mechanisms of correlation between cortical and motoneuronal activities and movement direction and other movement parameters. We show that the directional modulation of neuronal activity in the motor cortex and the spinal cord may result from direction-specific dynamics of muscle lengths. Our model suggests that directional modulation first emerges at the level of muscle forces, augments at the motoneuron level, and further increases at the level of the motor cortex due to the dependence of frictional forces in the joints, contractility of the muscles and afferent feedback on muscle lengths and/or velocities.</p></div

Distribution of preferred directions.

<p>Preferred directions (PDs) of cortical neurons (<b>A</b>) and Ia afferents (<b>B</b>) are fairly uniformly distributed over 360º. Colored lines show averaged PDs of cortical neurons and Ia afferents (FBIa) for corresponding muscles. The gray areas around each line represent standard deviation across 50 simulations with randomly chosen torque distribution parameter values. The length of each vector reflects the index of directional modulation. PDs of antagonist flexor and extensor related neurons and feedback are 180º apart. PDs of cortical activity are similar to the PDs of the antagonist Ia feedback. Please note that the afferents are independent of the torque distribution and have the same PD for all 50 simulations.</p

Model structure.

<p><b>(A)</b> The motor cortex sends motor signals to the spinal cord neuronal network which sends its outputs to the muscles. The spinal cord combines motor signals with afferent feedback to generate the motorneuron outputs. <b>(B)</b> Organization of interconnections between Renshaw cells (RC), motoneurons (MN) and other interneurons in the spinal cord network. Motoneurons send their outputs to their corresponding arm muscles. Ia and Ib inputs are the feedback signals from the muscles. <b>(C)</b> A 2-joint arm with six muscles: four major flexor and extensor muscles about the shoulder and elbow joints, and two biarticular muscles controlling both shoulder and elbow joints. SF, EF and BF represent shoulder, elbow and biarticular flexors, and SE, EE and BE represent shoulder, elbow and biarticular extensors, respectively.</p

Rotating the workspace by an angle shifts the preferred directions in the same direction by the same angle.

<p><b>(A)</b> Rotation of the workspace about the shoulder. The default workspace (center) was rotated by 45º counter-clockwise (Left), or by 45º clockwise (Right). <b>(B)</b> The distribution of preferred directions for the three workspaces corresponding to panel <b>A</b>. Vector lengths represent directional modulation index (see text for details).</p